Method for correction of phase-contrast magnetic resonance imaging data using a neural network

ABSTRACT

A method is disclosed for phase contrast magnetic resonance imaging (MRI) comprising: acquiring phase contrast 3D spatiotemporal MRI image data; inputing the 3D spatiotemporal MRI image data to a three-dimensional spatiotemporal convolutional neural network to produce a phase unwrapping estimate; generating from the phase unwrapping estimate an integer number of wraps per pixel; and combining the integer number of wraps per pixel with the phase contrast 3D spatiotemporal MRI image data to produce final output.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims priority from U.S. Provisional PatentApplication 62/821559 filed Mar. 21, 2019, which is incorporated hereinby reference.

STATEMENT REGARDING FEDERALLY SPONSORED RESEARCH OR DEVELOPMENT

This invention was made with Government support under contracts EB026136and EB009690 awarded by the National Institutes of Health. TheGovernment has certain rights in the invention.

FIELD OF THE INVENTION

The present invention relates generally to methods for magneticresonance imaging (MRI). More specifically, it relates to methods forphase-contrast magnetic resonance imaging (MRI).

BACKGROUND OF THE INVENTION

Phase-contrast magnetic resonance imaging (MRI) encodes tissue velocityinformation into the phase of the acquired image data, allowing foraccurate quantification of blood flow for a variety of clinicalapplications.

Specifically, cardiac 4D flow MRI enables retrospective valve tracking,velocity profiling, and advanced flow pattern visualization. Theseevaluation techniques are used to identify symptoms of valvular heartdisease, such as stenoses and regurgitation, and congenital heartdisease, including shunting between regions of the heart.

At the time of acquisition, the velocity encoding parameter, v_(enc), isset, which determines the maximum velocity encoded in the phase, rangingfrom −π to π. Any velocities that exceed the v_(enc) will wrap andalias. Mismapped velocities can result in artifactual reversal of theflow vectors, making it difficult to accurately measure blood flow [6].A common clinical strategy is to preemptively overestimate the v_(enc)to prevent aliasing; however, there is an inverse relationship betweenv_(enc) and velocity-to-noise ratio (VNR). A robust phase unwrappingalgorithm would allow clinicians to choose a low v_(enc) and obtainuseable images with high dynamic range and VNR.

The phase aliasing problem is not unique to MRI nor an unexploredproblem. As described in [7], existing approaches vary indimensionality, application, and approach. The large majority ofapproaches are 2D, since many other applications of phase unwrappingonly require 2D data. Whether mathematical or deep learning-based, 2Dapproaches do not account for spatiotemporal correlations found inphase-contrast MRI.

Additionally, phase unwrapping algorithms from other fields do notgeneralize or translate well to phase-contrast MRI applications becausethey do not account for MRI-specific knowledge [1-3, 8-9]. Meanwhile,existing MRI-based techniques fail to provide a robust solutionaccounting for both spatial and temporal dimensions [4-5, 10-21].

BRIEF SUMMARY OF THE INVENTION

This description discloses a deep learning-based phase unwrapping modelto correct for phase aliasing. It exploits spatiotemporal correlationsunique to phase-contrast MRI data, trained only on data with simulatedphase aliasing.

A first model uses 3D convolutions to examine performance on bothretrospectively and prospectively acquired phase aliased data, and isthen extended to a (2+1)D with separated convolutions to demonstrateincreased efficiency without noticeable performance loss. Both networksare compared with a state-of-the-art single-step Laplacian approach

A major challenge with a deep learning-based approach is the run-timeefficiency. 4D flow MRI scans can produce up to 30,000 velocity imagesper exam. Therefore, an unwrapping approach needs to run efficiently inorder to process all velocity images within a clinically feasibletimeframe. To address this challenge, the present technique includes thefollowing features:

-   -   A three-dimensional deep spatiotemporal phase unwrapping model        that efficiently generates corrected velocity maps,    -   We utilize the generated velocity maps to calculate the integer        number of wraps for each slice and produce the unwrapped output,    -   A training procedure during which the network is shown images        with simulated low v_(enc) values, through which the network        learns to correct for aliasing.

In one aspect, the invention provides a method for phase contrastmagnetic resonance imaging (MRI), the method comprising: acquiring 3Dphase contrast spatiotemporal MRI image data; inputing the 3Dspatiotemporal MRI image data to a three-dimensional spatiotemporalconvolutional neural network to produce a phase unwrapping estimate;generating from the phase unwrapping estimate an integer number of wrapsper pixel; and combining the integer number of wraps per pixel with thephase contrast 3D spatiotemporal MRI image data to produce final output.

BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWINGS

FIG. 1. Flowchart demonstrating a processing pipeline for an embodimentof the present invention. Because the number of wraps should be aninteger value, the neural network output is used to calculate thatinteger number of wraps, which is added to the original input.

FIG. 2. The neural network architecture including three-dimensionalresidual learning blocks, according to an embodiment of the invention.Each block has two convolution layer and rectified linear unit pairs.The convolutions were padded to preserve the x- and y-dimension sizes.

FIG. 3. Images illustrating results of the techniques of the presentinvention. From top to bottom: original low v_(enc) data, output ofalgorithm in [14], output of our present method, high v_(enc) data. Bothalgorithms successfully unwrapped pixels in the descending aorta (whitearrow), while the present technique performed better in the aortic valve(grey arrow).

DETAILED DESCRIPTION OF THE INVENTION

According to one embodiment, the method feeds a 3D phase input (x, y,time) into a neural network, which produces an unwrapped estimate. Theinput is subtracted from the estimate, and the difference is rounded,thus calculating the integer number of wraps per pixel. Finally, thisnumber of wraps is added back to the input to produce the final output.

As an example implementation of this technique, as shown in FIG. 1,original 3D spatiotemporal MRI data 100 is input to a neural network 102which outputs a model estimate 104. The estimate 104 is used tocalculate in step 106 an integer number of wraps per pixel 108, which isthen combined with the original data 100 to produce final unwrappedoutput 110.

The neural network is trained on four-dimensional (4D) flow data andachieved efficient performance. With IRB approval, cardiac 4D flow datafor nine patients were acquired with a contrast-enhancedvelocity-encoded volumetric RF-spoiled gradient echo sequence and a highv_(enc) of 250 cm/s. Seven datasets were used for training, one forvalidation, and one for testing. Additional low v_(enc) data were lateradded for testing purposes. Velocity was calculated in three directionswith a four-point velocity encoding scheme, and phase wrapping wasretrospectively simulated into the data, with v_(enc) values between 25cm/s and 75 cm/s. Each set had between 200 and 240 slices along theaxial axis, but only the top third with the most velocity changes werechosen to eliminate slices that primarily have static tissue. Datasetsare split by slice and direction, resulting in 1,581 unique trainingexamples. During training, to minimize the amount of noise seen by themodel, the images were randomly cropped into patches in the spatialdomain, to five percent of the original size. Patches were chosen usingthe magnitude image as the probability density function because thecontrast enhanced blood typically maps to regions with highervelocities. Background regions, air, contain uniformly distributed phase[−π, π], which are not useful for the network to learn properly. Duringvalidation and testing, the data was not patched to evaluate performanceon full images.

The network architecture is shown in FIG. 2. The network has threeresidual learning blocks 200, 202, 204 [22], which use residualconnections 206, 208, 210 between convolutional layer pairs to expeditetraining convergence and mitigate vanishing gradient issues. The networkgenerates an estimated velocity map 212, from which the input 214 issubtracted.

The difference is rounded because the number of wraps must be an integervalue, and that quantity is added to the input to produce the finalunwrapped output.

For run-time efficiency, three-dimensional convolutions are converted topairs of two-dimensional and one-dimensional convolutions: one in thespatial domain and one in the temporal domain. Lower dimensionalconvolutions are better optimized in deep learning frameworks; thismethod sped up the inference time by 91.5%.

The neural network was designed after the Off-ResNet architecture, anintentionally shallow architecture with a large kernel, designed tolearn regional correlations well without overfitting to specificanatomical structures. 3D data (x, y, time) is fed into the network,producing an unwrapped estimate. The input is then subtracted from thenetwork output and rounded, producing the integer number of wraps perpixel.

Two networks were trained—one with 3D convolutions and one with 2Dconvolutions. Training data includes data from scans of 7 patients andwas acquired without phase aliasing. During training, randomized phasealiasing was simulated into the data, and images were cropped based onthe magnitude images, to focus on regions with higher likelihood ofaliasing. Both networks were evaluated with a retrospectively aliasedtest dataset, as well as prospectively acquired data. The results ofboth networks were compared with a state-of-the-art non-deeplearning-based Laplacian approach. Results were evaluated withnormalized root-mean-square error (NRMSE), structural similarity index(SSIM), and peak signal-to-noise ratio (PSNR).

The two convolutional neural networks are based on the Off-ResNetarchitecture. The Off-ResNet architecture first uses a singleconvolution layer 216, followed by three residual layers 200, 202, 204,and one final output convolution layer 218. All convolution layers are3D convolutions with a 5×5×5 kernel and 128 filters, and eachconvolution is followed by a rectified linear activation unit (ReLU)besides the output layer. The Off-ResNet architecture is intentionallyshallow, relative to other popular network architectures, to preservegeneralizability and avoid overfitting to specific anatomies. Wemodified the architecture to only input phase information, instead ofcomplex inputs in the original design. We experimented with addingmagnitude information inputs but did not see a major impact on theresults. We preserved the relatively large kernel size in our network,since we wanted the network to identify the spatial correlations overlarger regions and not just correlate spatially adjacent pixels.

For the first network we used 3D convolution layers, similar to theOff-ResNet design, but for the second network, we separated each 3Dconvolution into a 2D spatial convolution and a 1D temporal convolution.While we expected the decrease in parameters to affect performance, wehypothesized it could potentially help with increased generalizabilityand reduce risk of overfitting.

Dataset & Training

For training and retrospective evaluation, we acquired cardiac 4D flowdata for nine patients with a high v_(enc) of 250 cm s⁻¹. With IRBapproval, we used a contrast-enhanced velocity-encoded volumetricRF-spoiled gradient echo sequence on a 1.5 T scanner (GE) andreconstructed the data. Seven of the high v_(enc) datasets were used fortraining, one for validation, and one for testing. For prospectiveevaluation, the same sequence used to acquire the training, validation,and test data was also used to sequentially acquire two paired highvenc/low venc datasets from the same patient. Velocity was calculated inthree directions—anterior-posterior, right-left, andsuperior-inferior—with a four-point velocity encoding scheme.

During training, we mixed axial data from all three velocity-encodingdirections, in hopes our model could generally learn the features ofphase wrapped data, agnostic to direction. We retrospectively simulatedphase wrapping into the data by applying Eq. 1, where φ_(H) is highv_(enc) data, φ_(L) is simulated low v_(enc) data, and v_(enc,L) is thenew lower maximum velocity to encode in the data. We randomly generatedv_(enc,L) to be between 25 cm s⁻¹ and 75 cm s⁻¹ during training toaugment and increase our dataset. This range was heuristicallydetermined based on the low velocities of interest in cardiac 4D flowstudies, but other ranges could be selected for other applications.φ_(L)=φ_(H)−2 v _(enc,L)└φ_(H)/2v _(enc,L)+½┘  Eq. 1

Because there were far more non-wrapped pixels than wrapped pixels, webalanced the dataset by filtering and cropping slices. One-third of theslices of each patient were selected, by choosing the slices with thetop standard deviation of the average velocity over the cardiac phase.This eliminated slices with very little change in velocity, such asslices at the top and bottom that consisted primarily of cardiac muscletissue, rather than blood flow. After filtering through the slices, ourtraining set consisted of 1,640 slices.

Slices were cropped in the spatial domain for the network to focus on asmaller region. We chose the center of the cropped region with themagnitude image as the probability density function, since thecontrast-enhanced blood had the highest magnitudes, so regions withblood flow had a higher chance of being selected. By using thismagnitude-weighted stochastic cropping, the network spent less time onpurely noise inputs, which are found in anatomies filled with air inaxial 4D flow data.

We trained both networks for 1,000 epochs. The 3D network trained for112.74 hours, while the (2+1)D network trained for 32.48 hours. Bothnetworks used an L2 loss and were optimized with RMSprop optimizer, toavoid vanishing and exploding gradients.

Evaluation

We evaluated both neural network models on retrospectively aliased testdata and prospectively acquired high v_(enc)-low v_(enc) paired data. Weacquired the retrospective test data with a v_(enc) of 250 cm s⁻1 andwrapped it down to 25 cm s⁻¹, 50 cm s⁻1, and 75 cm s⁻1, to test bothnetworks on a similar task as the training data. We also ran theretrospective test data through a state-of-the-art single-step Laplacianapproach for comparison. All outputs were compared with the ground truthnon-wrapped phase data with normalized root-mean-square error (NRMSE),structural similarity index (SSIM), and peak signal-to-noise ratio(PSNR). Additionally, we manually segmented select vessels to compareaverage flow over time.

We also evaluated both networks against the baseline Laplacian method onprospectively acquired paired data. The v_(enc) data was run through allthree approaches and compared with the high-v_(enc) data.

Applications

This invention can be used for any and all MRI scans with relevantinformation encoded into the phase of the MRI signal. From adimensionality perspective, this method can be useful fortwo-dimensional cine phase-contrast MRI, displacement encoding (DENSE)MRI, four-dimensional cardiac flow MRI, and any other phase-based datawith x-, y-, and time dimensions. From an anatomy perspective, althoughthe network is trained on only axial cardiac data, this technique cangeneralize to other anatomies and scan planes if provided sufficienttraining data.

Our technique can also extend to non-MRI applications, since the networkcan learn spatiotemporal correlations unique to the application.

In comparison with traditional spatial phase unwrapping algorithms, thepresent invention can leverage previous MRI exam data to automaticallylearn to robustly perform phase unwrapping. Multi-dimensional phaseunwrapping problems are ill-conditioned [23] causing traditionalalgorithms to become unstable when wrapping artifacts are severe. Ourtechnique learns domain-specific correlations from training data toproduce a robust unwrapping algorithm that is able to overcome solutioninstability. We show this in FIG. 3 by comparing our method with one ofthe state-of-the-art optimization-based algorithms, on a datasetinitially acquired with a low v_(enc), of 80 cm/s. In comparison with anexisting deep phase unwrapping technique [11], our technique takes intoaccount the temporal dynamics.

It will be appreciated by those skilled in the art that the embodimentsdescribed above include many specifics for the purposes of illustrationonly, and that these details may be varied without departing from theprinciples of the invention as taught herein. For example, the trainingmay include various loss functions, including SSIM, l2 norm, l1 norm,and regularized combinations, all to tune performance.

This technique can be modified to include different neural networklayers, blocks, and architectures. This can include modifications toindividual layers, including using circular or symmetric convolutionsinstead of zero-padded convolutions; or incorporation of establishedneural network blocks, including: residual networks, U-Nets,autoencoders, recurrent neural networks, and fully connected networks.Regularization layers, like batch normalization and dropout, can also beadded.

The neural network may be generalized for different anatomies andanatomical planes, by extending the variety of anatomy in the trainingdataset. Because phase-contrast MRI for different anatomies varies inthe size of the temporal domain, we can also train our network on avariety of levels of sparsity in the time domain.

The techniques may be extend to correct for eddy current induced phaseartifacts by modeling and simulating the artifacts and retraining ourmodel.

Similarly, the technique may be extended to address dephasing artifactsthat occur low venc acquisitions when there is turbulent flow. Signaldephasing artifacts can be simulated by adding additional noise to thetraining data in regions with low magnitude signal.

REFERENCES

-   [1] C. Simopoulos and K. F. Ustuner, “Phase unwrapped velocity    display for ultrasound medical imaging,” U.S. Pat. No. 7,682,311, 23    Mar. 2010.-   [2] H. Liu, X. Zhao, and X. Li, “Method and system for regional    phase unwrapping with pattern-assisted correction,” U.S. Pat. No.    9,897,433, 20 Feb. 2018.-   [3] M. R. Lucas, “Method for phase unwrapping using confidence-based    rework,” U.S. Pat. No. 9,964,640, 8 May 2018.-   [4] M. A. Gulsun, M.-P. Jolly, and C. Guetter, “Phase unwrapping in    phase contrast MR imaging,” U.S. Pat. No. 9,689,949, 27 Jun. 2017.-   [5] Y. Ye and J. Zong, “Methods and apparatuses for phase    unwrapping,” U.S. Pat. 2018/0203087, 19 Jul. 2018.    References-   [6] K. S. Nayak, J.-F. Nielsen, M. A. Bernstein, M. Markl, P. D.    Gatehouse, R. M. Botnar, D. Saloner, C. Lorenz, H. Wen, B. S.    Hu, F. H. Epstein, J. N. Oshinski, and S. V. Raman, “Cardiovascular    magnetic resonance phase contrast imaging,” J. Cardiovasc. Magn.    Reson., vol. 17, no. 1, p. 71, December 2015.-   [7] M. Jenkinson, “Fast, automated, N-dimensional phase-unwrapping    algorithm,” Magn. Reson. Med., vol. 49, no. 1, pp. 193-197, January    2003.-   [8] L. Ying, “Phase unwrapping,” in Wiley Encyclopedia of Biomedical    Engineering, Hoboken, N.J., USA: John Wiley & Sons, Inc., 2006.-   [9] M. Costantini, F. Malvarosa, F. Minati, L. Pietranera, and G.    Milillo, “A three-dimensional phase unwrapping algorithm for    processing of multitemporal SAR interferometric measurements,” in    IEEE International Geoscience and Remote Sensing Symposium, 2002,    vol. 3, pp. 1741-1743.-   [10] M. Loecher and D. B. Ennis, “Velocity reconstruction with    nonconvex optimization for low-velocity-encoding phase-contrast    MRI,” Magn. Reson. Med., vol. 80, no. 1, pp. 42-52, July 2018.-   [11] K. M. Johnson, “3D velocimetry phase unwrapping using    block-wise classification with a shift variant fully 3D    convolutional neural network,” in ISMRM 26th Annual Meeting, 2018.-   [12] A. Justice, “Phase unwrapping of 4D-flow MRI data with graph    cuts,” 2018.-   [13] J. Cheng, Y. Mei, B. Liu, J. Guan, X. Liu, E. X. Wu, Q.    Feng, W. Chen, and Y. Feng, “A novel phase-unwrapping method based    on pixel clustering and local surface fitting with application to    Dixon water-fat MRI,” Magn. Reson. Med., vol. 79, no. 1, pp.    515-528, January 2018.-   [14] M. Loecher, E. Schrauben, K. M. Johnson, and O. Wieben, “Phase    unwrapping in 4D MR flow with a 4D single-step Laplacian    algorithm,” J. Magn. Reson. Imaging, vol. 43, pp. 833-842, 2016.-   [15] M. Untenberger, M. Hüllebrand, L. Tautz, A. A. Joseph, D.    Voit, K. D. Merboldt, and J. Frahm, “Spatiotemporal phase unwrapping    for real-time phase-contrast flow MRI,” Magn. Reson. Med., vol. 74,    no. 4, pp. 964-970, October 2015.-   [16] E. Barnhill, P. Kennedy, C. L. Johnson, M. Mada, and N.    Roberts, “Real-time 4D phase unwrapping applied to magnetic    resonance elastography,” Magn. Reson. Med., vol. 73, no. 6, pp.    2321-2331, June 2015.-   [17] J. Langley and Q. Zhao, “A model-based 3D phase unwrapping    algorithm using Gegenbauerpolynomials,” Phys. Med. Biol., vol. 54,    no. 17, pp. 5237-5252, September 2009.-   [18] H. Bagher-Ebadian, Q. Jiang, and J. R. Ewing, “A modified    Fourier-based phase unwrapping algorithm with an application to MRI    venography,” J. Magn. Reson. Imaging, vol. 27, no. 3, pp. 649-52,    March 2008.-   [19] M. F. Salfity, J. M. Huntley, M. J. Graves, O. Marklund, R.    Cusack, and D. A. Beauregard, “Extending the dynamic range of phase    contrast magnetic resonance velocity imaging using advanced    higher-dimensional phase unwrapping algorithms,” J. R. Soc.    Interface, vol. 3, no. 8, pp. 415-27, June 2006.-   [20] Q.-S. Xiang, “Temporal phase unwrapping for cine velocity    imaging,” J. Magn. Reson. Imaging, vol. 5, no. 5, pp. 529-534,    September 1995.-   [21] J. N. Ongori, E. M. Meintjes, and B. S. Spottiswoode, “3D phase    unwrapping of DENSE MRI images using region merging.”-   [22] K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning    for image recognition,” December 2015.-   [23] D. C. Ghiglia and M. D. Pritt, Two-dimensional phase    unwrapping: theory, algorithms, and software, 1st ed. New York    [etc.]: John Wiley & Sons, Inc., 1998.

The invention claimed is:
 1. A method for phase contrast magneticresonance imaging (MRI), the method comprising: acquiring phase contrast3D spatiotemporal MRI image data; inputing the 3D spatiotemporal MRIimage data to a three-dimensional spatiotemporal convolutional neuralnetwork to produce a phase unwrapping estimate; generating from thephase unwrapping estimate an integer number of wraps per pixel;combining the integer number of wraps per pixel with the phase contrast3D spatiotemporal MRI image data to produce final output.